News from TREC

As TREC enters its 25th year, it might seem that we’re quite far from our “adhoc search” roots, but a closer look reveals that we are pushing IR research in new and different directions by expanding the Cranfield model, not abandoning it. The TREC 2015 tracks were quite novel and presented significant challenges for participants. The Live QA track requires participants to answer open-domain questions from Yahoo Answers in real time. In the fourth year of the Contextual Suggestion track, a live task was added to the existing batch experiment, with feedback from Mechanical Turk workers. The new Total Recall track (motto: “I’ll be back”) asks participating systems to achieve high recall with a simulated human in the loop. The new Dynamic Domain track also featured relevance feedback from a simulated user as the system navigates topics with multiple diverse sub-intents. The fifth year of the Microblog track moved the task to a real-time scenario, with participants running on the public feed of tweets as they happened. In the third iteration of the Temporal Summarization task, participants systems emitted sentence updates about emerging events and were penalized for latency as well as irrelevance. The new Tasks track featured both task understanding and task completion components. Lastly, the Clinical Decision Support track required participants to return biomedical journal articles relevant to a patient case narrative with respect to a specific question such as diagnosis or testing.